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Abstract 

Cumulative weight enumerators of random linear codes are introduced, 
their asymptotic properties are studied, and very sharp thresholds are 
exhibited; as a consequence, it is shown that the asymptotic Gilbert- 
Varshamov bound is a very sharp threshold point for the density of the 
linear codes whose relative distance is greater than a given positive num- 
ber. For arbitrary random codes, similar settings and results are exhibited; 
in particular, the very sharp threshold point for the density of the codes 
whose relative distance is greater than a given positive number is located 
at half the asymptotic Gilbert- Varshamov bound. 



1 Introduction 

Let F be an alphabet for coding with cardinality \F\ = q > 1. Any non-empty 

subset C of F" is called a code of length n; the fraction R{C) = ^"^^^'"^ is 
said to be the rate of the code C. Codewords, i.e. the elements of C, may be 
changed at some probability due to noise when they are communicated through 
a channel. Shannon |14j introduced the capacity of the channel and showed that: 
if a number r is below the capacity then there are codes of rate r and decoding 
rules such that the probability of recovering the codewords goes to 1 as n — t- oo; 
however, if r is beyond the capacity then for any code of rate r the probability 
of recovering the codewords is bounded from above by a number less than 1. 
For the latter case, Wolfowitz [18] further showed that for any code of rate r 
the probability of recovering the codewords goes to as n — J> oo. Following 
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terminology from physics, Shannon's capacity is a phase transition point, also 
known as a threshold point, for the channel from high fidelity to no fidelity. 

Hamming [6] introduced the Hamming distance to construct error-correcting 
codes. We denote the minimum Hamming distance of any code C C by 
dniC); denote the fraction '^"^^^^ by A(C), and call it the relative distance 
of C. An important question is: how to get the highest rate of the codes in F" 
with relative distance greater than a given positive number 51 Gilbert and 
Varshamov [16j exhibited a function in 5 and n, called the Gilbert- Varshamov 
bound, to bound from below this highest rate, cf. [71 §2.8, §2.9]. The limit as 
n — >■ cx) of this lower bound, which we denote by GY q{S), is called the asymptotic 
Gilbert- Varshamov bound, or GV-bound in short. The GV-bound coincides ex- 
actly with Shannon's capacity of the q-ary symmetric channel, so it is also called 
the Gilbert- Shannon- Varshamov bound. When is a finite field, there have been 
several families of linear codes in [TSl |51 [T71 13] whose parameters improve the 
Gilbert- Varshamov bound (as a function in S and n). In the asymptotic sense, 
the family in [15] does exceed the GV-bound but only for the case when g > 49. 

For a random linear code of rate r, Varshamov in [16| proved that: if 
r < GVq{S), then the probability of the relative distance of Lr being greater 
than 6, denoted by Pr (A(Lr) > (5), goes to 1 as n — s- oo. Note that the 
relative distance A(Lr) of the random linear code Lj. of rate r is a random 
variable. Pierce [13] proved that A{Lr) is asymptotically distributed at one 
point Sovir), where SQy{r) denotes the solution of the equation r = GYq{x) in 
the variable x and is called the GV-distance. From this result of Pierce, it could 
further follow that: if r > GVq{S) then Pr (A(L^) > S) goes to as n — > oo. In 
other words, the GV-bound GVq{6) is a threshold point from high density to 
extreme sparsity of the linear codes of rate r whose relative distance is greater 
than a given 6. 

For a binary random code C from a so-called random code ensemble, the 
distance enumerator with respect to a given codeword, i.e. the number of the 
codewords with a given distance from the given codeword, has been extensively 
studied; cf. 0], (3, ^Q\- Barg and Forney [1] investigated, by means of Cheby- 
chev's inequality, the distance enumerator and showed that the relative distance 
A(C) is asymptotically distributed at the point <5Gv(2r). Similar to the linear 
case but not as precisely as what Pierce [13] did, it could follow that ^GYq{S) 
is a threshold point for the density of the codes of rate r whose relative distance 
is greater than a given 6. 

In this paper we consider a more general setting with a more sensitive sec- 
ond moment method, including the Cauchy inequality and the Paley-Zygmund 
inequality (see |T2]). For a random linear code L of length n and rate r we 
define the cumulative weight enumerator to be the number of codewords whose 
weight is bounded from above, and study its asymptotic property. We exhibit 
very sharp threshold points for the distributions of the cumulative weight enu- 
merator. It is just a special case of the thresholds we obtain that the GV-bound 
GVq{S) is a very sharp threshold point for the density of the linear codes of 
rate r whose relative distance is greater than a given S. Furthermore, the result 
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of [13] is also a special case of the threshold points for the cumulative weight 
enumerators. 

For any random codes over an alphabet with q (not necessarily equal to 2) 
elements we consider a similar setting and obtain similar sharp threshold points. 
As immediate consequences, we show that ^GY q{5) is a very sharp threshold 
point for the density of the codes of rate r whose relative distance is greater 
than a given S; on the other hand, we point out that the asymptotic distribution 
of the relative distances of the random codes with given rate r is exactly located 
at (5Gv(2r), as described in [1 . 

The paper is composed of five sections. After the introduction, in Section 2 
the necessary notations are introduced for quotation. In Section 3 random linear 
codes are discussed; while in Section 4 arbitrary random codes are studied. 
Finally we draw conclusions in Section 5. 



2 The KL-divergence and the GV-bound 

Throughout the paper, F is an alphabet with cardinality |i^|=(7>l, nisa 
positive integer; and we always take the following parameters: 

6 e {0, 6o) where (5o = 1 - (7"\ d = 6n; r G (0, 1), k ^ rn; (2.1) 

where d = Sn should be understood, in practical computations, to be the integer 
nearest to Sn; and it is the same for k — rn. 

In this paper, exp(a;) = denotes the natural exponential function with the 
natural base e, while exp^(a;) = denotes the exponential function with base q; 
and logqX denotes the logarithm function to the base q, while \nx denotes the 
natural logarithm function. Recall that, for any < p,p' < 1, the g-ary KL- 
divergence (or information divergence) of p to p' is defined as 

DMp')=P^ogqfr + {l-p)\ogq^ 

with the conventions log^ = and log^ § = 0. It is a convex non-negative 
function in p with a unique zero point aX p ~ p' , see [111 §1.2]. 
We consider the following partial sum of binomials: 

MS) - E (fjShil - <5o)"-^ = E (")'^o(l - ^o)"~' , (2.2) 

1=0 ^ ^ j=0 ^ 

where (") denotes the binomial coefficient. We have the following estimation 
which will play a key role in the asymptotic analysis of random codes: 

Pn{S) = exp^ {-nD,{S\\So) - ^ log^ n + p„(<5)) (2.3) 

with /3„((5), n = 1, 2, • • • , being functions on S such that 

Jim exp, {p,,iS)) = 
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We may also write 

/3„(<5) = exp, {-nD,{S\\So) - \ log, n + 0(1)) . (2.3') 

The estimation (I2.3P is somewhat known, for example, [13l Formula (7)] is 
just a special case where q = 2. We sketch a proof of (|2.3|) for completeness. 

Proof of (2.3). Let T, = (^)5S(1 - ^0)""^ in particular, = {sl)d^o"i^ " 
5o)^^"''^". By Stirling's formula, nl = (27m)^n"e-"+''(") with < j{n) < 

SO we have an estimation: 

12n ' 

Ta= {27rdil-S)nyK^p^{-nD,{S\\do)-Oi^)). (2.4) 
For any index i with < i < c? we have 

Ti n-i+1 ' So n-i ' 5o Ti + i ' 

For d = dn we let 

"~ - (n-d+l)5o ^ 1-5+^ ' ^ ' 

further, for 6' with < S' < S and rf' = S'n, we let 



ut — 

Then 



n = -^^—^ = r- 



and 



i + 

thus 
that is, 

— ^ (l-a)-iTd ^ 

For any £ > we can take 5' < 5 such that < e; hence > 1 — e 

this way we can get that 



m 



n->-oo (^1 — a) '^Id 

Finally, combining (j2.4p . (1^3)) and (1^^ . we obtain the estimation ([O)) . □ 
As an immediate consequence of (2.3'), we get the following lemma. 
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Lemma 2.1. Let notations he as in (2.1) and (2.2). Then 

^/3„(<5) =exp,((r-i?,(5||5o))ri-ilog,n + 0(l)). (2.7) 
In particular, 

lim g™/3„(J) = < (2.8) 
[w, r>Dq(5\\5o)- 

the limit is exponentially convergent for every r G (0, 1) except for r = Dq{5\\5{,); 
at the exceptional point it converges to at the rate of going to 0. 

Recall that any a = (ai, • • • ,a„) £ F" is said to be a word of length n 
over i*"; the Hamming distance between the words a and a', denoted by dn^SL, a'), 
is defined to be the number of the indices i where a and a' have different 
coordinates a; ^ a[. By B{a, 6n) we mean the Hamming ball centered at a S F" 
with radius Sn. The volume of the Hamming ball is denoted by Vq{n,6n) = 
\B{a,6n)\. With the notation (|2.2[) . we have 

Vq{n,6n) = Eto C)!'? - 1)' = l''MS). (2.9) 

Any non-empty subset C of F" is called a q-ary code C of length n over F, 
any c G C is said to be a codeword. We can write the cardinality of C as 
Id = where i?(C) = isM^ll is called the rate of C. The minimum 

distance of C, denoted by dniC), is defined as duiC) = min d/f(c, c'); and 

the fraction '^"f^^ is called the relative distance of C, denoted by A(C). 

For a given 5 G (0,(5o), a longstanding important question is the following: 
what is the highest rate R{C) for C C F" with A(C) > 51 We sketch it as a 
remark. 

Remark 2.1. Applying a typical greedy algorithm to select codewords step 
by step, we can get a code C C F" with A(C) > 5 such that IJ i3(c, 5n) ~ F", 

hence |C| • Vq(n, 5n) > q"; by Eqns (gS]) and (2.3') we have that 

'Z^(^)" > y£j;r) = exp, {nDq{5\\5^) + i log, n - 0(1)) (2.10) 

where y is just the usual Gilbert- Varshamov bound (cf. [7, §2.8, §2.9]), 

which is a function in n and S; thus 

R{C)>DMSo) + '-¥~0{^), (2.11) 

which implies that 

• for large enough n, the highest rate of the codes of F" whose relative 
distance is greater than 5 is larger than Dq{5\\5{)). 

A code C is said to be linear if F is a finite field and C is a subspace of 
the vector space F". For linear codes, applying a similar greedy algorithm to 
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construct parity check matrices of linear codes, we can get the same conclusions 
as above. 

From the asymptotic perspective, the quantity Dq{5\\5Q) is known as the 
asymptotic Gilbert- Varshamov bound for g-ary codes at the relative distance 5. 
We denote it by GVq{S) = Z3g((5||(5o), and call it the GV-boundin short. In fact, 
the GV-bound is also related to the so-called q-ary entropy function 

Hq{S) = S\ogq{q - 1) - Slogq S-{l-S) log,(l - ,5) 

in the sense that GV,((5) = 1 - Hq{S), see [3 §2.10.6]. 

As we have pointed out for the KL-divergence, GVq{S) is a convex and 
strictly decreasing function in 5 e (0, (5o) with GVq(O) — 1 and GYq{do) — 0. 

Remark 2.2. In the following, for convenience, we let rs = GVq{S) for 
any S G (0, (5o), and tq ~ 1, rs„ = 0; i.e. rg is the GV-bound at distance S. On 
the other hand, because of the monotone property of the function GVq{S) in 
6 € (0, ^o)i for any r G (0, 1) there is a unique real number in the interval (0, (5o), 
which we denote by Sr, such that r = GYq{5r); of course, i5o = 1 — as fixed 
in (|2.ip . and Si = 0. The quantity Sr is called the GV-distance at rate r; see 
references [J, [TU], [H §6.2]. 

3 Random linear codes 

In this section, we keep the parameters in (|2.ip and assume that is a finite field 
of order q, hence i^" is a vector space over F of dimension n. The Hamming 
weight Wfiia) for a G F" is defined by wh{sl) = (i/f(a, 0), where denotes 
the zero vector. Any linear subspace L of the vector space F" is said to be a 
linear code; the rate of L is R{L) = '^'"^ , where dimL denotes the dimension 
of the linear code L; while the relative distance of L is A(L) = "'""J , where 
wh(L) ~ min wh(c) is called the minimum weight of the linear code L. 

O^cGL 

Consider the probability space Cr{F") whose sample space is the set of all 
linear codes of rate r in and whose probability function is equiprobable. 
Take Lr G CriF"), i-e. Lr is a random linear code of length n and rate r. Set 
A/"" (S) to be the cardinality of the set of the non-zero codewords whose weight 
is at most Sn, i.e. 

J^2AS)^\{Lr\{0})nB{0,Sn)\, (3.1) 

where ir\{0} denotes the set difference. In particular, the minimal 6 such 
that Af^ (S) > 1 is just the relative distance A{Lr) of Lr- We remark that 

Sn 

i^) ~ Til -^L 0)' where Af^ (j) stands for the number of codewords c such 
j=i 

that dniOyC) = j, and is called the distance enumerator with respect to the 
codeword 0; cf. [U §B] or [TTl §6.2]. We name 7^2^(5) the cumulative weight 
enumerator. 
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For asymptotic analysis, we are concerned with the fraction IZ^aj^Lid^^ which 
is called the growth rate of J\f2^{S), and abbreviated by R(J\f2^{S)y, i.e. 

Afl{S)^exp^[n-R{j{f2^{6))). (3.2) 

Note that Af^ (S) and its growth rate are random variables over the probability 
space CriF"). By Eqn < Af2^{S) < q"' - 1, so 

e {-oo}U [0,r)- (3.3) 

Recalling from Remark 2.2 that rg — GYq{6) denotes the GV-bound at the 
distance (5, we state the main result of this section. 

Theorem 3.1. Let notations be as in (3.2]) . Assume that 6 £ {0,So), 
re (0,1) andt e {-oo}U[0,r). Then 



lim Pt (r{N'2^X6)) >t 



j 1, if t < r - rs; 
\o, if t>r-rs; 

the limit converges exponentially for any t G { — cxd} U [0, r) except for t — r ~rs; 
at the exceptional point it converges to at the rate of going to 0. 

Proof. Let C{F'',F") be the probability space whose sample space is the 
set of all injective linear maps / : F'^ — F" and every sample is equipped with 
equiprobability. For / e C{F'',F") the image 

Lf^f{F'')^{f{h)\heF''}CF^ (3.4) 

is a linear code of rate r in F^, and any £ ^r{F^) can be obtained in 
this way; further, Lj — Lf ioi f,f' e C{F^,F^) if and only if /' — fg for a 
g G GL{F'') where GL(F'^) denotes the set of the linear automorphisms of F^. 
In a word, the random linear code L f and L^. have the same random property 
though they are defined in different probability spaces; in particular 



Pr(R{MlM >t) ^Vr(R{Ml^{5)) 



> t 



Thus, instead of Lr, we prove the theorem for the random linear code Lf for 

/ e C{F^,F-^). 

First we wrtite the random variable Af^i ('^) ^ ^ sum of Bernoulli random 
variables. For each b e F'"'\{0}, we have the following indicator random variable 

Xo.^l^' 0<--(/(b))<^"; (3.5) 
lO, otherwise. 

Since /(b) runs uniformly over the vectors of i^"\{0} when / runs uniformly 
over £(F'^, f "■), the expectation of Xo.b is easy to calculate (cf. Eqn 



E(Xo,b) = Pr (Xo,b = 1) = "^-y,"^'^ = Pn{5) - 0{q-n- (3.6) 
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Further, we construct a non-negative integral random variable 

Xo= E ^o,b. (3.7) 

beF''\{0} 



It is obvious that Xq = Af^j i^) = cxp^ [n ■ R{NI^ [5]) j , and R{J^^^ {5)) > t is 
equivalent to Xq > g*". 

As the second step of the proof, we compute the expectation of the random 
variable Xq. By the linearity of expectations and Eqn p.6|) we have 

E(Xo) = J2 E(Xo.b) -(g™-l)(/3„(5)-0(<Z-")) 

beF''\{0} 

= 9™/3„(<5)(l - g-™) - g™0(g-")(l - g^™); 

thus, from Eqn (j2.7l) . we obtain the following conclusion which we write as a 
lemma for later reference. 



Lemma 3.1. E(Xo) exp, ((r - r5)n - i log, n + 0(1)) - 0(g('-i)"); 
[O, r<rs; 
I oo, r > rg. 



particular, lim E(Xo) 



The third step of the proof of Theorem 3.1 is proving the following lemma. 
Lemma 3.2. If r > rs, then lim J-, °'L — 1. 

Proof. We fix a bi G F'^\{0}. For any b G F'^\{0} we have seen from 
Eqn (|3.6p that E(Xo.b) = E(Xo.bi); and it is also obvious that 

E(Xo|Xo,b = 1) = E(Xo|Xo.b, = 1), 

where E(Xo|Xo_b = 1) denotes the conditional expectation of Xq when the 
event "Xq i, — 1" occurs. By the lemma of [H Appendix A] we have (or, one 
can check it directly): 

E(Xo)^ _ ^ E(Xo,b) _ (q'-"-i).E(Xo.bJ _ E(Xo) ,o o\ 

~ j^^^Vy^^j E(Xo|Xo,b = l) E(Xo|Xo,bi=l) E(Xo|Xo,bi=l) ' ^■^■°> 

By the linearity of expectations, we have 

E{Xo\Xo.b, = 1) = E E(Xo,b|Xo,bi = 1). 

be_Ffe\{0} 

For b e F'''\{0}, there are two cases. 

Case 1: b and bi arc linearly independent. Since the event "A'o.bi = 1" 
occurs, wc consider the injective linear maps f : ^ F" which satisfy that 
/(bi) G B{Q,5n). When such / is randomly selected, the image /(b) runs 
over the vectors in \-F/(bi) uniformly at random, where Ffih^) is the 1- 
dimensional subspace of F" generated by /(bi). Note that F/(bi) C i?(0, 5n) 



8 



since any non-zero vector of Ff{hi) has weight (/(bi)). Therefore, we get 
that 



E(Xo,b|Xo,b, = 1) = Pr(Xo,b - l|Xo,b, = 1) = ^i^p^ 



where the last equality follows from Eqn p.6p . 

Case ^: b and bi are linearly dependent. Then /(b) and /(bi) are linearly 
dependent, hence ^//(/(b)) = ^^(/(bi)); so E(Xo,b|-'«^o,bi = 1) = 1. 

Since the number of the non-zero b which are linearly dependent on bi is 
equal to g — 1, from Eqn (|3.8p we get that 



E(Xo> ~ E(Xo) ^ Ep?^ 



E(Jfg) _ bEFfc\{0} ^ (g-l) + (g'°-?)E(Xo.bJ 



Note that ^,^^^^"^4 > 1 (by Cauchy's inequality), and that {q — gr)E(Xo,bi) < 
{q'' - l)E(Xo,bJ = E(Xo) (by 1^). We get that 

^ - E(Xo)2 ^ E(Xo) + 

Since r > rg, by Lemma 3.1 we have that lim E(Xo) = oo; thus we get the 

n— ^oo 

desired conclusion lim J^, °.L —1. □ 

n->-oo E(,AoJ 

Now we can complete the proof of Theorem 3.1. 

Recall that Pr [R{Mf^^{5)) > = Pr {Xq > g*"). Et>r~rs, then, by 
Markov's inequality and Lemma 3.1, we have that 

Pr (Xo > g*") < ^ < expg ((r ^ rs - t)n ~ ^ log, n + 0(1)) ; 

since r — rs — t < 0, we get lim Pr [Xq > q*") = 0. 

n— >-oo 

In the case when t < r — rg, we write g*" = 9 ■ F,{Xo) where 
i = ^ = exp, ((r -rs-t)n-^ log, n + 0(1)) - 0( exp,((r - i - l)n)) ; 

since r — — t > and r — rg — t > r — rg — 1, we have lim 9^0. By the 
Paley-Zygmund inequality (see [12 ) and Lemma 3.2, we get that 

lim Pr(Xo > q*") = lim Pr (Xo > 9E{Xo)) > lim (1 - 0)'^^ = 1. 

Finally, in all the above arguments (including the arguments for Lemma 3.2), 
the limits are exponentially convergent except for t = r — rg; at the exceptional 
point, the limit in Eqn (12. 8p of Lemma 2.1 converges to at the rate of ^ 
going to 0. We are done for the proof of Theorem 3.1. □ 
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Note that r-te (0,r]U{oo} fort G {-cx3}U[0,r). Since GYq{5) is a strictly 
decreasing function for 6 G [0,(5o]i with the notations in Remark 2.2 we have: 

t < r — rs rs < r — t ■^=^ S > 6r~t ; 

t > r — rs rg > r — t S < Sr~t ; 

where Sr-t G [0, Sq) with the following convention: 

6r-t = if r-t^oo. (3.9) 

Thus we can rewrite Theorem 3.1 as follows. 

Theorem 3.1*. Let notations be as in Theorem 3.1. With the convention 
in hS. 9\) we have that 



lim Vr[R{Afy {5)) > t 

n^oo 



0, if 5<5r-u 

1, if (5 > 5r-u 



the limit converges exponentially for any 5 G (0,(5o) except for 5 = 5r-t; o,t the 
exceptional point it converges to at the rate of going to 0. 

If we take t = Q and fix r, then the statement "i?(A/'°^((5)) > 0" is just saying 

that "A/'i^CfJ) > 1"; and "A^°^(^) > 1" implies that the relative distance satisfies 
A(Lr) < 5. As a consequence of Theorem 3.1* we obtain again the main result 
of [13] at once as follows. 

Corollary 3.1. Assume that r G (0,1) is given and Sr G (0;(5o) is the 
corresponding GV- distance (see Remark 2.2). Then 

lim Pr {A{Lr) < 6) = 




the limit converges exponentially for any S G (0, (5o) except for 5 = Sr; at the 
exceptional point it converges to at the rate of going to 0. 

Remark 3.1. For any given r G (0, 1), A{Lr) is a random variable defined 
over the probability space Cr{F"'); and Pr (A(Lr) < 5) is just the cumulative 
distribution function of A{Lr). The above corollary shows that, asymptotically 
speaking, the distribution of A{Lr) is concentrated at the GV-distance 6r; more 
precisely, for arbitrarily small positive real number e we have 

lim Pr (A{Lr) < Sr) = ^ lim Pr (A(i^) >Sr + e), 

which is the main result of 1131. 

We now turn to Theorem 3.1. Recalling that the growth rate of the cumu- 
lative weight enumerator i?(A/'° (S)) is a random variable over Cr{F'^), we can 



10 



view Theorem 3.1 as a characterization of its distribution, i.e. for arbitrarily 
small £ > we have 



lim Pr (R{AfL (S)) < r - - e) = = lim Pr (i?(A/'° (6)) >r- rs); (3.10) 

that is, R{Af2 (S)) is asymptotically distributed at exactly one point r — rs- On 
the other hand, if we take t = Q again but fix 6, then, since "AA^ [5) < 1" is 
equivalent to "A(Lr) > <5", we get a consequence from Theorem 3.1: 

Corollary 3.2. Assume that d £ (0,(5o) is given and rs G (0,1) is the 
corresponding GV-bound (see Remark 2.2). Then 



lim Pr (A(L^) > 5) 



1, if r < rs\ 
0, if r > rs; 



the limit converges exponentially for any r G (0,1) except for r — rs; at the 
exceptional point it converges to at the rate of going to 0. 

The above corollary implies that the GV-bound rs ~ GWq [5) is a very sharp 
threshold point of the parameter r such that the density of the linear codes with 
relative distance greater than 5 changes from highly dense to extremely sparse. 

Remark 3.2. More generally, for any point [6, r) G [0, Jq] x [0, 1] we can ask 
whether or not there exist linear codes Lr of rate r such that A(Lr) > 5. Our 
results show an asymptotically probabilistic answer as follows. The rectangle 
[0, 5o] X [0, 1] can be divided into three areas as shown in Figure 3.1: 




Area I: surrounded by the (5-axis, the r-axis and the curve r = GW q{5) (the 
curve itself included), where every point (^, r) satisfies lim Pr (A(Lr) > 5) — 1; 

n— f oo 

in other words, "A(Lr) > ^" is nearly a certain event. This fact is well known 
since Varshamov |16j . 
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Area II: surrounded by the curve r = GY q{5) and any upper bound 
(the dotted curve m Figure 3.1); for any point {S,r) in this area, we have 
Urn Pr (A{Lr) > 5) = Q; that is, in the probabihstic sense, "A(Lr) > S" is 

nearly an impossible event. Note that, for a point {6,r) of this area, hnear 
codes with rate r and with relative distance > S may exist, e.g. see the works 
[ini ISl [T71 [3] , see also Remark 2.1. However, in the asymptotic sense, the limits 
(as n — >■ oo) of the parameters of the linear codes in [SI [T71 [3] are located on 
the curve r = GVq{d), hence in Area I, while the reference [H] does construct 
linear codes with parameters in Area II for the case where q > 49. 

Area III: above any upper bound, where "A(ir) > (5" is an impossible 
event. There exist many upper bounds, for example, five upper bounds are 
described in [71 §2.10] (but the exact upper bound is still unknown). Our result 
also gives an explanation why it is so hard to determine the exact boundary of 
the area where "A(ir) > S" is an impossible event: differently from the GV- 
bound where a very sharp phase transition occurs, the process from very small 
probability to zero probability is a very flat and calm stream, hence it seems 
hard to locate accurately the boundary from small to zero. 

Remark 3.3. Instead of the injective linear maps from F'' to as in 
Eqn p.4p . one can consider the probability space consisting of all the linear 
maps / : F'' — > F" and with equiprobability. This probability space is just the 
so-called random linear code ensemble in Shannon's information theory, cf. [Ill 
§6.6]. Then, for the random linear codes f{F'') C F", all the results of this 
section still hold. 

4 Arbitrary random codes 

In this section, the parameters in (|2.ip are still preserved, but F is just an alpha- 
bet. Let Cr be a random code of rate r in F", in other words, we consider the 
probability space Cr{F") consisting of all codes of rate r in F" with equiproba- 
bility. The issues investigated in this section are similar to the last section, but 
the computations and the analysis of random dependence are different. 
Similarly to Eqn p.ip . for c G we define 

Sn 

J^cJS) = \{Cr\{c})nB{c,6n)\ = E M^Jj), (4.1) 

where (j) stands for the number of codewords c' such that (i^f(c, c') = j, 
and is called the distance enumerator with respect to the codeword c. Further, 
we set 

Sn 

which is the number of 2-subsets {c, c'} of Cr (i.e. subsets of Cr with cardinality 
2) such that (i/f(c, c') < 5n. Thus, we name Mc-ri^) the cumulative distance 
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enumerator. In particular, the minimal 5 such that Mc^{^) > 1 is just the 
relative distance A(Cr) of Cr- Similar to Eqn (13.21) . we are concerned with the 

growth rate R{SfcA5)) = i^l^^^, i.e. 

McM = exp, (n ■ R{AfcAS))) ■ (4.3) 

Note that Afc^ (S) and its growth rate are random variables over the probability 
space Cr(F"). From Eqn (g^) we have that < Afc, (S) < £21sZ^- differently 
from Eqn (|33| . the range of R(Afc^{S)) is 

R{J^cAS)) e {"Oo}U [0,2r). (4.4) 



Theorem 4.1. Let notations be as in ^.3^ . Assume that 5 S (0, ^o) 
re (0,1) and t e {-oo} U [0,2r). Then 



lim Pr {RiNrJS)] >t 

n— ^OO 



1, if t < 2r - rs; 
0, if t > 2r - rs; 



the limit converges exponentially for any t G {— oo}U[0, 2r) except for t = 2r—rs; 
at the exceptional point it converges to at the rate of going to 0. 

Similarly to the proof of Theorem 3.1, instead of Cr, we consider 

Cg = {5(b) I b e F^-} C (4.5) 

for g eC{F'',F"), where C(F'=, F") is the probability space of all injective maps 
from F'' to F" (recall that k — rn) with equiprobability, since we have 

Pr (r{NcM) >t)= Pr (r{McAS)) > t) . 



We first reconstruct the integral random variable Afc {S) with Bernoulli vari- 
ables. For any h ^h' G F'^ we have the following indicator random variable 

rbb' = l^' '^Hi9{h),giW))<Sn; 
I 0, otherwise. 



Since g is injective, we get the expectation as follows 

(n,(5n) 



E(lb.b') = Pr {Y^M = 1) = = l3n{S) - 0{q-n- (4.7) 



Remark 4.1. For b G F we can define the integral random variable 

i"b- E i"b,b' (4.8) 

b'eF''\{b} 
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so that Yb = AfQ^\S). Similar to what we did in Section 3, one can check that 
the conclusions of Lemma 3.1, Lemma 3.2 and Theorem 3.1 are all still true for 
Yb, i.e. 

(i) . E(yb) -expJ(r-r5)n-ilog^n-0(g('-i)")); 

(ii) . if r > rs, then lim ^^Is = 1; 

9(b)^A^ ^ „trA\ _ Jl. t<r-rs; 



(iii). lim Pr (r(M^\S) > g*")) 



0, t > r — rg. 



However, in order to characterize J\fcg{S), we need to introduce further an 
integral random variable: 

Y=h E Yh= E i"b,b', (4.9) 
where E means that {b,b'} runs over the 2-subsets of F''. Then we have 

b^b'SF'' 

that Y = Afc,{S) = exp^ (n ■ i?(7V'c, (5))) , and the event "i?(A/'c« ((5)) > t 
equivalent to the event "F > g*"" . 



IS 



Lemma 4.1. E{Y) = exp^ ((2r - rs)n - \ log, n + 0(1)) - ©(g^^'-i)"); 
'O, 2r<r5; 
oo, 2r > rs- 



in particular, lim E(F) 



Proof. Fix a 2-subset {bi, ba} of F''. Let K = i(j™(g™ - 1). Let {b, b'} 
be an arbitrary 2-subset of F'^. By (|4.7p we have E(Yb,b') = E(Ybi.b2); further, 
by the linearity of expectations and Eqn (|4.9p , we have 



E(r) = K ■ E(rbi,bJ = if • (PniS) - o(g-")). 

As KPn{S) = ^q^™l3n{S){l ~ q^"''), the first equality of the lemma follows from 
Lemma 2.1. Then, it is clear that lim E(y) = if 2r < rg; on the other hand, 

n— ^oo 

since 2r — > 2r — 1, we have that hm E(y) = oo if 2r > rs- □ 

n— >-C30 

Lemma 4.2. // 2r > rg, then lim pfrTrr = 1. 

Proof. Keep the notations as in the proof of Lemma 4.1. Besides the equal- 
ity E(yb,b') = E(ybi,b2), it is also obvious that E(r|Yb,b' = 1) = E(y|Ybi,b2 = 
1). Similar to the proof of Lemma 3.2, we have: 

E(y)' ^ V E(yb.b') _ -g-E(ybi.b2) _ E(y) 
i^T^ E(y|Yb,b'=i) E(y|ibi,b2=i) E(y|Yb,,b2=i)' 

and 

E(r|yb„b2 = 1) = E E(yb,b'|>b„b2 = !)• 
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For an arbitrary 2-subset {b,b'} of F'^, there are three cases. 

Case 1: {b,b'} = {bi,b2}, then E(yb,b' |>b,,b. = 1) = 1. 

Case 2: {b,b'} and {bi,b2} have one vector in common, the number of 
such 2-subsets is 2{q^"^ — 2); for this case, we have 

E(rb,b'|i^b„b. = 1) - ^-y,"^^'' < ^"y,"^'^ = E(yb„bJ. 

Case 3: {b, b'} n {bi,b2} = 0. Since g is injective, we can bound the 
conditional expectation as fohows 

E(rb.bHi-b,.b. = 1) < < = E(rb,.b.). 

By Eqn (HJU)) we have (recah that K ■ E(Ybi,b2) = E(F)): 



E(y^) _ E(r|yb^.b,=i) ^ i+(A:-i)E(yb,,b,) ^ i 

E(y)^ ~ E(Y) — E(y) E(F) 



Because 2r > r^, by Lemma 4.1 we have hm E(y) — oo. Thus we get the 

n— f oo 

desired conclusion hm wfv^ = 1- CH 

Recahing that ''R{Nc^{5)) > t" is equivalent to "y > g*"" (see Eqn gH)), 
we prove Theorem 4.1 now. 

Proof of Theorem 4.1. First assume that t > 2r — rg. By Markov's 
inequality and Lemma 4.1, we have that 

Pr (Y > g*") < ^ < exp^ ((2r - t - rs)n - i log^ n + 0(1)) ; 

since 2r — t ~ rs < 0, we get lim Pr {Y > g*") = 0. 

Next, we assume that t < 2r — rg. We write g*" = 6* • E(y) where 

l = W= ^^P? ((^'^ - t - n)n - i log, n + 0(1)) - 0( exp,((2r - t - l)n)) ; 

since 2r — i — > 2r — i — 1, we have that lim = 0; by the Paley-Zygmund 
inequality (see [12]) and Lemma 4.2, we get that 

lim Pr(y > g*") = lim Pr iY > 6IE(y)) > lim (1 - Of^^Q^ = 1. 

Finally, in all the arguments above (including the arguments for Lemma 4.2), 
the limits are exponentially convergent except for t = 2r — r^; at the exceptional 
point, the limit in (|2.8p of Lemma 2.1 converges to at the rate of going to 
0. We are done for the proof. □ 

Note that 2r - t e (0, 2r] U {oo} for t e {-oo} U [0, 2r). Since GVg(J) is a 
strictly decreasing function for S G [0, So], we have 

t < 2r — rs <S=^ rs < 2r - t ^==^ S > S2r-t ; 
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t>2r-rs rs>2r-t <^ S < S2r-t ; 

where the GV-distance S2r-t £ [0, Sq) with the fohowing convention: 

S2r-t = if l<2r-t. (4.11) 

Similar to Theorem 3.1*, we can rewrite Theorem 4.1 as follows. 

Theorem 4.1*. Let notations be as in Theorem 4-1- With the convention 
in 1^. j j[ ) we have that 



lim PT{R(J\fcJS)) >t] = 

n—¥oo 



0, if (5 < S2r-U 

1, if (5 > 62r-t; 



the limit converges exponentially for any S G (0, Sq) except for S = 52r-tj o,t the 
exceptional point it converges to at the rate of going to 0. 

If we take t — Q and fix r, then the statement "i?(A/'(7^(5)) > 0" is none 

other than ''NcM > 1"; and "7\/'a(5) > 1" is equivalent to "A(a) < S'\ we 
get a straightforward consequence of Theorem 4.1* as follows. 

Corollary 4.1. Assume that r G (0,1) is given and S2r G [0, (5o) "is the 
GV-distance at the rate 2r with the convention in |^T7Ip. Then 



lim Pr (A(a) < S) 



0, S<62r; 

1, S > S2r; 



the limit converges exponentially for any S G (0,(5o) except for S — 52r', o,t the 
exceptional point it converges to at the rate of going to 0. 

Remark 4.2. Similar to the explanation in Remark 3.1, A{Cr) is a random 
variable over the probability space Cr{F"), and Pr (A(Cr) < S) is its cumulative 
distribution function. The corollary exhibits that the distribution of A(Cr) 
is asymptotically concentrated at the GV-distance d2r', more precisely, for an 
arbitrarily small positive real number e we have 

lim Pr (A{Cr) < S2r) = = lim Pr (A{Cr) > + e). 

n— foo n— >-oo 

The references [1] , [10] have also described the distribution, but in terms of the 
distance enumerator with respect to any given codeword, and not as precisely 
as above. 

Wc turn to Theorem 4.1. Similar to Eqn (|3.10|) . R{J\fc^{S)) is a random 
variable over Cr{F^), and for an arbitrarily small e > we have 

lim Pt {R{AfcriS)) < 2r-rs-e) = = lim Pr {R{Afc,XS)) > 2r~rs); (4.12) 
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that is, RiAfc^iS)) is asymptotically distributed at exactly one point 2r — rg- 
On the other hand, if we take t = again but fix J, then, since "Ac;r(^) = 
(i.e. Mcr{^) < 1)" is equivalent to "A(Cr) > 5", we get a corollary as follows. 



Corollary 4.2. Assume that 5 £ (0,(5o) is given and rs G 
corresponding GV-bound. Then 



(0,1) 



the 



lim Pr {A{Cr) > 6) 



'l, if r < ^rs; 
^0, if r > ^rs; 



the limit converges exponentially for any r G (0, 1) except for r = ^rs; at the 
exceptional point it converges to at the rate of going to 0. 

Remark 4.3. The above corollary says that ^rs = |GVg((5) is a very sharp 
phase transition point for the density of the codes whose relative distance is 
greater than the given number 5. Moreover, similar to Remark 3.2, for any point 
{6, r) of the rectangle (0, Sq) x (0, 1) we can consider the asymptotic behavior of 
the probability Pr (A(Cr-) > 5) for the random code Cr- The rectangle can be 
divided into three areas by the curve r = ^G\q{5) (not the curve r = GVg(^)!) 
and the curve for any upper bound as in Figure 4.1: 

Area I: surrounded by the ^-axis, the r-axis and the curve r = \GYq{5) 
(the curve itself included), where "A(Ci.) > (J" is nearly a certain event. 

Area II: surrounded by the curve r = ^GYq{5) and any upper bound (the 
dotted curve), where "A(Cr) > 8" is a nearly impossible event. 

Area III: above any upper bound (the dotted curve), where "A(Cr) > 5" 
is an impossible event. 




half GV-bound 



&0 5 

Figure 4.1 Three areas for the probability of the event "A(Cr-) > (5" 



Remark 4.4. For an arbitrary random code Cr of rate r in F", Area II of 

Figure 4.1 can be further divided by the curve r = GYq{5) into two subarcas 
II' and II", where the subarea II' is surrounded by the curve r = \GYq{5) and 
the curve r = GY q{5) (the curve itself included), and the subarea II" is the 
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remaining part of Area II; see Figure 4.1. The two subareas are distinct from 
each other in the fohowing two points of view. 

• For any (r, 5) in the subarea II', though "A(Cr) > (J" is nearly impossible, 
by the greedy algorithm one can construct codes of rate r and relative distance 
> 6; see Remark 2.1. On the other hand, the greedy algorithm does not work 
in the subarea II". 

• For the variable Yb — ■^c'^\S) defined in Eqn (14. 8p . from Remark 4.1(iii) 
one can, by taking t — 0, get that 



where the range r < rs — GY q{S) covers Area I and the subarea II'. Comparing 
it with Corollary 4.2, one can see that the cumulative effect does change the 
threshold points. 

Remark 4.5. Instead of the injective maps from F'^ to F" as in Eqn (|4.5p . 
one can consider the probability space whose sample space consists of all the 
maps g : F^ F^ and every sample is equipped with equiprobability; this 
probability space is just the so-called random code ensemble in Shannon's infor- 
mation theory, cf. [TTl §6.1]. Then, for the random code g{F'') C F'^, all the 
results of this section still hold. 

5 Conclusion 

For a random code C and c G C, the number of codewords with distance j from 
c, denoted by Af^ij) and called the distance enumerator with respect to c, has 
been extensively studied with second moment analysis. In this paper we have 
introduced the cumulative distance enumerators of random codes, and investi- 
gated their thresholds by more sensitive second moment methods, including the 
Paley-Zygmund inequality. We have shown that the threshold phenomena of 
the relative distance and the GV-bound are just special extreme cases of the 
threshold phenomena of the cumulative distance enumerators. 

For a random linear code L of length n and rate r over a finite field with q 
elements and a number S with < S < 1 — g~^, we defined A/'°(i5) = X]^=i -^lU) 

and its growth rate i?(A/'{'((5)) = jj^ve proved that the growth 

rate is asymptotically distributed at exactly one point r — rs, where rs denotes 
the GV-bound at S, see p.lOp . Two consequences, which are more or less already 
known, were directly derived: the relative distance A(L) of L is asymptotically 
distributed at the GV-distance Sr; while the GV-bound rs is a sharp threshold 
point for the density of linear codes whose relative distance is greater than S. 

We consider an arbitrary random code C of length n and rate r over an 
alphabet with q elements in a similar manner. For a number S with < S < 



1 — q ^, we defined A/'c('5) = 5 X^cec -'^c(.?)' proved that the growth 
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rate R{J\fc{S)) = „^ — - is asymptotically distributed at exactly one point 
2r — rs, see (|4.12p : and deduced directly two consequences: the relative distance 
A(C) is asymptotically distributed at the GV-distance S2r', half the GV-bound 
^rs is a sharp threshold point for the density of codes whose relative distance 
is greater than d. 
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